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ABSTRACT/KEYWORDS 

This article presents the results of a national study of 39 higher education institutions that collected 
information about the cost measures used to evaluate faculty development for online teaching as well as 
anticipated future decisions regarding faculty development activities and content in a budget-cutting 
situation. Generally, institutions are more likely to expand low marginal cost training (e.g., online 
modules) and eliminate training content dealing with tools (e.g., Facebook/Twitter). The two most 
popular cost measures are related to dollars per hour of training provided per academic year and dollars 
spent per new online course per academic year. 
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I. INTRODUCTION 

What can we learn from faculty developers about how they understand and assess the costs and cost- 
effectiveness of what they do? Given the current budget difficulties of states and the growing pressure on 
many institutions to live within their means (if not also to increase the dollars available to them), we need 
to know if and how the costs of faculty development are assessed, how potential budget cuts might alter 
current offerings, and what this implies about the perceived value of the training. 

Faculty development plays a role in the Sloan Consortium’s [1] five pillars of quality. Faculty 
satisfaction is one of those pillars, which stresses the importance of faculty satisfaction with the online 
teaching experience and the faculty’s commitment to improving what they do in their online courses. 
Sloan-C characterizes faculty satisfaction as resulting from institutional support (also rewards and 
involvement in governance), which in turn is defined as the opportunity for “training in online 
instructional skills” (1, 5). Thus, faculty development in online teaching is a critical foundation for 
quality online education and justified the creation by Sloan-C of an advisory panel of practitioners and 
researchers focused on faculty satisfaction, development, and support 
( http://sloanconsortium.org/ialn advisory panel fs) . As a first step in the work of the advisory panel, it 
was critical to identify the current state of knowledge for faculty development for online teaching [ 1 ]. A 
second step was collection of data from a survey of institutions and analysis of those data to answer 
questions about cost and cost-effectiveness. 
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II. REVIEW OF LITERATURE 


A. Why Cost? 

The interest in faculty development for online teaching is due, in part, to a number of changes happening 
to the higher education landscape. First, based on changes in state funding of public higher education 
from FY09 to FY10, 28 states appropriated monies 0.1 to 10.0 percent less year-over-year, 10 states 
appropriated monies 10.1 to 22.0 percent less, and 12 states appropriated the same or larger monies year- 
over-year [3]. From FY10 to FY11, 17 states appropriated monies that were 0.1 to 10.0 percent less year- 
over-year, 15 states appropriated monies 10.1 to 22.0 percent less, and 18 more states appropriated the 
same or larger monies year-over-year [3]. In the last fiscal year (2012-2013), 30 states increased their 
appropriations to higher education (including institutional support and financial aid) [4]. While these facts 
may imply an improvement in state budget situations and signal an increase in funding for higher 
education, the growing demands on states for K-12 improvement, health care reform, and other pressing 
concerns undermine such an interpretation. This funding outlook for higher education is supported by a 
Moody’s Investor Service report [5], which notes all revenue streams traditionally supporting higher 
education to be under economic, technological, and public pressure. In addition, these changes appear to 
be out of institutions’ hands, forcing leaders to be more strategic and innovative in their efforts to improve 
institutional productivity, develop new markets and services, prioritize use of resources, and demonstrate 
value to those who fund higher education. 

Second, demands on higher education institutions have increased regarding access for traditional-aged 
college students as well as adults seeking additional education or job retraining. Employers have 
increased their demands for appropriately-trained employees and the public has demanded better 
accountability from higher education in addition to higher quality and lower costs [6]. Furthermore, 
President Obama focused attention on the country’s education system when he set a goal to “have the 
highest proportion of students graduating from college in the world by 2020” [7: 11]. To increase 
educational attainment while also increasing access in lieu of anticipated budget constraints requires 
higher education leaders to use resources more strategically and in a more productive manner. 

Third, preceding these events, states have become interested in using technologies to increase access to 
higher education for underserved (or poorly served) populations, and to improve higher education 
productivity in general. The growing interest in productivity is due to the confluence of three factors: 
constrained state budgets, increased numbers of high school graduates, and increased demand for college- 
level training by individuals seeking retraining and businesses seeking trained personnel (see [8] for a 
review of these factors). 

Many technologies have been promoted as a means to improve higher education’s productivity and 
quality [9, 10]. Distance education—including MOOCs—has been promoted as a way for states to serve 
more students at the same or lower cost. While this part of the promise seems not to have transpired, 
states retain an interest in lowering the cost per student served, while pushing institutions to engage in 
more innovative delivery systems. 

B. Cost-Efficiency and Cost-Effectiveness 

In the economic literature, there is an important difference between cost-efficiency and cost-effectiveness. 
For our puiposes here, cost-efficiency is defined as the provision of education at either less cost or more 
students at the same cost. Cost-effectiveness is defined as the provision of more or better learning at the 
same or lower cost. Put more simply, efficiency tends to emphasize more quantity while effectiveness 
includes an emphasis on greater quality. Unfortunately, many individuals confuse the two terms or use 
them interchangeably; these will be the definitions used in this study. 

C. Research on Cost, Technologies, and Faculty 

This issue has benefited from studies conducted as part of the Program in Course Redesign hosted by the 
National Center for Academic Transformation led by Carol Twigg [11, 12, 13, 14]. In their initial efforts, 
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thirty institutions were funded to redesign courses while enacting plans to lower costs and document 
improved student learning. On average, these institutions reduced costs by 37%—some projects reduced 
costs by 15%, others by 77%—saving $3.1 million per year in operating costs [14]. This effort continues 
to expand and now includes projects in many states. 

In a review of over 60 studies on cost-efficiencies and/or cost-effectiveness of online learning [15], a 
variety of institutional and/or program-based studies were documented and analyzed. The studies looked 
at changes in faculty roles, instructional approaches, and organizational models among other variables to 
determine which, if any, increased efficiency (the provision of education at either less cost or more 
students at the same cost) or productivity (the provision of more or better student learning at the same or 
lower cost). Studies were classified by variables focusing on faculty, content, or instructional approach 
(see [16] for a justification of this framework) with findings analyzed across variables. 

Based on this work, Meyer [15] abstracted four major principles for achieving productivity through online 
learning These principles include (1) substituting high-cost labor with technology (through online 
modules, self-paced learning, automated grading), (2) substituting high-cost labor with lower-cost labor 
(through use of peer tutors, adjuncts, graduate assistants), and (3) substituting technology for capital 
(through online learning, there is less demand for capital space which can slow the demand for new 
buildings). However, it is important to note that none of these “substitutions” can be achieved without 
redesigning courses to maintain or improve student learning. Therefore, the fourth principle for achieving 
cost-effectiveness through online learning is the incorporation of robust instructional design principles. 
Instructional design is a large field of study which draws upon various learning theories to design, 
develop, implement, and evaluate learning experiences or materials, which may be online or not. 

We can look at cost in two ways: first, as faculty productivity and second, as student learning 
productivity. Three studies help demonstrate how faculty create productivity for themselves as well as for 
students through online learning. 

In 2009-10, ten experienced online faculty were interviewed to elicit examples of how they improved 
student learning productivity in their online courses [17]. Student learning productivity was defined as 
more, faster, or better learning based on data available to the individual faculty being interviewed. The ten 
faculty represented nine different states and thirteen different fields or disciplines. All faculty were 
tenured or on a tenure-track at a master’s or doctoral level institution of public higher education. Based on 
a thematic analysis of the examples given, improvement in student learning occurred through six different 
processes. First, faculty increased student access to content through various online venues (including 
online libraries, websites, journals, tools, and games). Second, faculty designed courses that would, in 
effect, change their role in the class. This had two parts: it increased access to faculty by email, discussion 
boards, Skype, or other web-based tools; and it changed faculty roles to “a guide on the side” rather than 
the content provider. Third, faculty increased interaction with and among students. Fourth, faculty 
emphasized active student participation by choosing assignments requiring student effort (including use 
of experiential learning, group work, learning to learn, and feedback). Fifth, faculty connected students to 
the “real world” by designing assignments with real-world applications or involving real-world players. 
Sixth, faculty focused on developing and communicating clear and definite uses of time. These findings 
suggest that faculty can and do find ways to use different tools in different ways to improve student 
learning productivity. This study was repeated in 2010-2011 with 11 experienced community college 
faculty members who were also interviewed to elicit examples of how they improved student learning 
productivity in their online courses [18]. These 11 faculty members represented eight different states, 
nine different fields or disciplines, and were all permanent or full-time faculty members at community 
colleges in southern or western regions of the United States. Based on a thematic analysis of the 
examples given, improvement in student learning occurred by 1) emphasizing seven approaches to 
increasing student engagement with learning for the course, 2) using six different visual, verbal, and 
pacing tactics that would focus student attention on learning, 3) using assessment techniques to improve 
learning, and 4) pursuing a personal passion for online teaching. 
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The focus of interviews with the ten experienced online faculty at the master’s/doctoral institutions also 
included asking how teaching online had affected their teaching and research productivity [19]. Based on 
a thematic analysis of interview transcripts, several faculty felt their teaching productivity had increased, 
both as a result of personal choices made during the design or delivery of instruction (such as 
emphasizing providing extensive course content online rather than having faculty provide content through 
online lectures) and an increase in efficiency (teaching more students in a class). They knew the 
importance of instructional design as well as addressing student and program needs. Several faculty had 
freed up their professional time by teaching online, which was then spent on various service or research 
activities although this was dependent on the stage of the faculty person’s career. At first, early stage 
faculty had more difficulty learning how to free up their time, but once freed, used the time for research 
activities; later stage faculty used their free time for research and service activities. Service activities 
included serving on an external service board, serving as department chair, or serving as chair of the 
institution’s Faculty Senate. 

D. The Connection to Faculty Development 

This research on cost, productivity, and instructional design brings us to an obvious connection to the 
need for faculty development for online teaching. While such faculty development seeks to help faculty 
teach well online, those faculty can, in turn, affect an institution’s cost-efficiency or cost-effectiveness. 
Therefore, it is intriguing that the literature on faculty development for online teaching so far lacks a 
consideration of cost-effectiveness of faculty development offerings [1]. Certainly, this would be a 
difficult undertaking requiring that cost information be defined, collected, and reported. However, as 
institutional budgets are becoming more constrained by a decline in state funding, competition within 
internal parts of an institution for resources, as well as demands on institutions for services and 
accountability are increasing. This means that assessing cost-effectiveness of faculty development is 
becoming essential to improving the effectiveness of an institution. How can faculty developers justify 
their activities if they cannot provide a good estimate of a) the cost of each effort, b) the efficiency of that 
effort (defined by the number of outcomes produced for the cost), and c) the quality of those outcomes for 
the cost? Of course, undertaking such a calculation involves several issues raised in [1]: a better 
understanding of the outcomes produced by faculty development initiatives as well as a way to evaluate 
all aspects of faculty development efforts. With better attention to these elements, faculty developers may 
one day be able to say that while a workshop costs less, one-on-one consulting provides more pertinent 
changes to online courses. Or that efforts emphasizing transformative learning may be very costly, but 
produce more secondary and tertiary student learning outcomes than a a simpler workshop on preparing a 
podcast, wiki, or blog. While the latter may still be essential, perhaps less costly ways of providing these 
skills can be explored. In any case, the final result would be empowering faculty developers with the data 
and insights to make the best choices regarding desired outcomes and available budget. 

This article builds upon the findings in [20] which presented the results of a survey of institutions that 
offer faculty development for online teaching. The results in [20] focused first and foremost on the 
frequency that institutions used the training activities and content used in this study, but also evaluated the 
value of each separate training activity and content. However, in the current study, institutions assessed 
what they would do if budget cuts at their institution forced them to change what they do. In other words, 
this analysis looks more closely on the value-and-cost assessment of faculty developers of the same 
training activities and content profiled in [20] as well as possible outcome measures that assess the cost of 
faculty development for online teaching, including one specific measure: how many faculty were served 
in the most recent academic year. 

E. Research Questions 

Four research questions guide this study. First, do institutions use the cost of training in their evaluations 
of faculty development? Second, if forced to cut budgets, what training activities or training content 
would institutions expand, keep, scale back, or eliminate? Third, what measures of cost-efficiency or cost- 
effectiveness do institutions use? Fourth, what percent of full-time versus part-time faculty were served 
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by faculty development? This last question was included in an attempt to collect actual data from 
institutions on their ability to serve a portion of their total faculty members. 

II. METHODOLOGY 
A. Research Design and Instrument 

This study is based on survey research that collected information from participating higher education 
institutions. As this is one of the first attempts to assess faculty development for online teaching practices 
in a national sample, survey research is an appropriate approach. 

The instrument used was developed by the author and was based on a thorough review of the published 
literature on faculty development for online teaching [1], Once a draft of the instrument was made, the 
instrument was reviewed by the Sloan-C Advisory Panel for Faculty Satisfaction as well as 
representatives of the Sloan-C and WCET organizations, including organizational leaders and researchers, 
faculty developers, and faculty who conduct research on this topic. Considering that this would be a 
national study of faculty development for online teaching, the findings woidd be of interest to members in 
both organizations. This process resulted in many additions and revisions that resulted in a cleaner and 
more comprehensive instrument. Given the face validity of the items, it is likely that the data resulting 
from the instrument are valid and reliable. 

This study used five items from the instrument (the complete instrument contained 26 items). Appendix A 
provides the exact wording for the five items used in this study. The items dealt with the four matters. 
First, the institution indicated whether it used the cost of training in its evaluation of faculty development. 
Four answers were provided, and the labels (and the number codes) are Yes (3), No (2), Don’t Know (1), 
and May Use in Future (0). 

Second, the institution answered two questions, “Suppose that you were forced to cut your unit’s budget, 
how would you adjust your training activities?” and “Suppose that you were forced to cut your unit’s 
budget, how would you adjust your training content?” These questions were chosen to assess the value 
placed by the institution on training in the context of reduced resources. The training activities included 
enewsletter initiatives, one-on-one sessions, hands-on sessions, short sessions (<2 hours), workshops (2-5 
hours), one-time sessions, multiple sessions (2-5 sessions), many sessions (>5 sessions), summer semester 
training, year-long training, consortial (multi-institutional) training, webinars, online modules, peer 
training, train-the-trainer, create online course, peer review of course, and use of instructional design 
students. Training content included CMS (course management system) platforms, blogs, wikis, podcasts, 
Facebook/Twitter, mobile pedagogies, advanced topics for experienced faculty, problem-based learning, 
experiential learning, critical thinking, use of case studies, creating community, student learning styles, 
Community of Inquiry, blended instruction, discipline-specific training, assessment of student learning, 
instructional design models, research base(s) of online learning, and externally-provided training. The 
five options (and numeric coding values) offered were Expand (4), Keep (3), Scale Back (2), Eliminate 
(1), and Do Not Offer (0). 

Third, the institution was surveyed on measures used to assess cost-efficiency or cost-effectiveness of 
faculty development. The options were (in each case “$” refers to faculty development dollars): “$ spent 
per headcount faculty served per academic year, $ spent per hours of training provided per academic year, 
$ spent per new online courses produced per academic year, $ spent per changes made to F2F classes per 
academic year, $ spent per measure of student learning (e.g., GPA) per academic year.” The first three 
measures are clearly cost-efficiency measures and last two measures more related to quality, and 
therefore, cost-effectiveness measures. The options for answers were Yes (2) and No (1). 

Fourth, the institution was asked to provide 1) the total number (unduplicated headcount) of a) tenured/ 
tenure-track faculty and b) the number of tenure/tenure-track faculty provided faculty development for 
online teaching in AY 2011-12 and 2) total number (unduplicated headcount) of a) part- 
time/adjunct/instructionaFprovisional faculty at the institution and b) the number of part- 
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time/adjunct/instructional/provisional faculty provided faculty development in AY 2011-12. 

B. Population and Sample 

A request to complete the survey was sent from a Sloan-C officer to the official representatives of higher 
education institutions that are members of the Sloan-C organization, or 407 institutions, which includes 
both higher education institution and businesses. The author also sent a request to complete the survey to 
the online WCET Discussion Board, which is open to any individual who is an employee of a W1CHE 
Cooperative for Educational Telecommunications (WCET) member organization, or 295 members, 
including both higher education institutions and businesses. The request asked that the survey information 
be forwarded to the individual responsible for faculty development at the institution. Because institutions 
can and are members of both organizations, the request asked institutions to beware of duplicative emails 
and two items were included in the instrument (institution name and individual’s name) so that 
duplicative responses could be identified and one eliminated. Also, it is difficult to accurately assess the 
response rate because many higher education institutions belong to both organizations or because they 
may not have specific programs that provide faculty development for online teaching. 

Responses were received from 39 institutions including 13 research/doctoral institutions, 12 master’s 
institutions, three baccalaureate institutions, and 11 associate-level institutions. The survey was completed 
by individuals located in Academic Affairs Offices (57.9%), Chief Information (Technology) Offices 
(23.7%), academic departments (18.4%), academic colleges (13.2%), or Central/System Offices (5.3%). 
Respondents’ titles included coordinator, director, dean, and Vice President. This diversity of job titles 
and locations seem to imply that faculty development for online teaching is occurring in many different 
locations at the institution and under different guises or names, but is happening nevertheless. 

C. Data Collection 

The instrument was created within SurveyMonkey.com, which provides a flexible set of question types 
for the researcher and long-term data storage. The initial request to institutions for responses to the 
instrument was sent January 4, 2013 and the deadline for receipt of responses was February 1, 2013. On 
this date, the survey was closed to further responses and analysis began. 

D. Data Analysis 

For research question 1, “Do institutions use the cost of training in their evaluations of faculty 
development?” the answer is reported as a frequency and percent of respondents with experience 
evaluating the cost of training or a desire to do so in the future. 

For research question 2, “If forced to cut budgets, what training activities or training content would 
institutions expand, keep, scale back, or eliminate?” answers are reported as a percent of respondents 
specifying whether the activity or content would be expanded, kept, scaled back, or eliminated. 

For research question 3, “What measures of cost-efficiency or cost-effectiveness do institutions use?” 
answers are simply reported by frequency and percent of respondents using the measure. 

For research question 4, “What percent of full-time versus part-time faculty were served by faculty 
development?” responses were grouped into full-time and part-time categories and reported as a percent 
of faculty served. 

III. FINDINGS 


A. Cost of Training 

The simple answer to the question, “Do institutions use the cost of training in their evaluations of faculty 
development?” is “not many.” Eight of the 39 institutions responding to the survey (or 21% of 
respondents) do evaluate the cost of their faculty development for online teaching; three additional 
institutions indicated their interest to assess the cost of training in the future. This information captures 
how infrequently the sample institutions appear to attend to cost of faculty training. 
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B. Response to Budget Cuts 

For research question 2, “If forced to cut budgets, what activities would expand, be kept, scale back, or 
eliminate?” answers are reported in Table 1 as the percentage of institutions responding to specific 
activities; percentages are based only on those institutions that claim to use this activity in their training 
(not all institutions in the sample). 


Table 1 

Percent of Institutions Indicating Actions To Be Taken to Training Activity If Budget Cut 
(n=varies) _ 


Activity 

Expand 

Keep 

Scale back 

Eliminate 

Enewsletter 

31 

58 

8 

4 

One-on-one training 

8 

49 

36 

8 

Hands-on training (e.g., in lab) 

6 

64 

31 

0 

Short session (<2 hrs) 

11 

68 

21 

0 

Workshop (2-5 hrs) 

0 

55 

36 

9 

One-time session 

0 

68 

18 

13 

Multiple sessions (2-5 sessions) 

6 

42 

39 

13 

Many sessions (>5 sessions) 

3 

27 

43 

23 

Summer semester training 

3 

45 

39 

13 

Year-long training 

4 

30 

43 

23 

Consortial training 

33 

33 

20 

13 

Webinars 

38 

21 

32 

9 

Online modules 

58 

34 

8 

0 

Peer training 

48 

41 

7 

3 

Train the trainer 

40 

50 

5 

5 

Create online course 

41 

54 

5 

0 

Peer review of course 

24 

62 

14 

0 

Use of instructional design 
students 

29 

29 

21 

21 


There are some interesting changes reflected in these figures. Generally, respondents plan to expand 
training activities that may be of marginal cost (e.g., enewsletter, online modules) once they have been 
developed or where costs may be shared with others (consortial training, webinars). In other words, these 
are training types where adding more participants does not markedly add to the cost of the training. 
Activities that may draw upon others who are not faculty development staff (peer training, train the 
trainer) will also be expanded while those training activities that may draw heavily upon staff (multiple 
sessions, many sessions, year-long training) would be scaled back or eliminated by some. 

What may be implied by the responses in Table 1 are that institutional assessments of training, and the 
value of such training, extend beyond cost alone. Hands-on training was favorably viewed as was one-on- 
one training (both of which include resources such as staff time). This, perhaps, reflects the value placed 
on direct instructional opportunities by faculty and developers alike. On the other hand, the expansion of 
short workshops and one-time sessions was not indicated, which may mean that these development 
opportunities are seen as less effective given their “one-time” nature of training, although this is 
conjecture. It is not conjecture, however, to say that having faculty create an online course and undergo 
peer review of that course has greater value in the eyes of these institutions and will be kept and expanded 
should a budget cut be on the horizon. 


Journal of Asynchronous Learning Networks - Vol 18, No 1 (2014) 





Table 2 

Percent of Institutions Indicating Actions To Be Taken to Training Content If Budget Cut 
(n=varies) _____ 


Content 

Expand 

Keep 

Scale back 

Eliminate 

CMS 

10 

71 

16 

3 

Blogs 

0 

45 

50 

40 

Wikis 

0 

24 

44 

32 

Podcasts 

9 

27 

27 

36 

Facebook/Twitter 

25 

21 

29 

25 

Mobile technologies 

25 

50 

29 

13 

Problem-based learning 

7 

71 

18 

4 

Experiential learning 

11 

63 

15 

7 

Critical thinking 

7 

74 

15 

4 

Use of case studies 

7 

70 

22 

4 

Creating community 

15 

74 

12 

3 

Student learning styles 

3 

66 

24 

7 

Community of Inquiry 

13 

48 

30 

9 

Blended instruction 

24 

65 

6 

6 

Discipline-specific training 

28 

40 

24 

8 

Assessment of student learning 

19 

72 

6 

3 

Instructional design model(s) 

9 

52 

30 

9 

Research base(s) of online 
learning 

4 

44 

48 

4 

Access to externally-run training 

17 

9 

48 

26 


Asking this question about what would happen in the case of budget cuts is a way to elicit institutions’ 
evaluations about different training activities (Table 1) or training content (Table 2). In other words, 
activities or content that developers see value in are likely to be kept or expanded in the face of budget 
cuts, while those that are less valued may be scaled back or eliminated. What is less clear is whether the 
value developers apply to these items are based on 1) the effectiveness of the activities or content in 
teaching faculty what they need to know or 2) their cost-effectiveness. Second, as noted in [20], there is 
evidence that faculty developers have a preference to keep or expand training that deals with pedagogies 
(e.g., experiential learning, critical thinking). On the other hand, training dealing with tools (e.g., blogs, 
wikis, Facebook/Twitter) is more likely to be scaled back or even eliminated altogether. Second, there is a 
lack of confidence or commitment to providing faculty with the research bases of online learning (also 
noted in [20]), which survey responses suggest would be scaled back or eliminated by over half of the 
institutions. Third, the institutions’ response to “access to externally-run training” is intriguing, since it 
captures both a slight expansion for some institutions as well as a larger scaling back for others. Perhaps 
this is simply a cost decision, with some institutions viewing external training options to be cheaper 
alternatives than in-house initiatives. In this situation, faculty developers are assessing externally-run 
training with a shaip eye on cost—businesses providing these types of training opportunities may need to 
take note. 

C. Cost Measures Used 

For research question 3, “What measures of cost-efficiency or cost-effectiveness do institutions use?” 
answers are reported by frequency and percent of respondents choosing the measure, along with the rank 
order of responses. 

Table 3 

Frequency and Percent of Institutions Using Various Cost Measures (n=38 institutions) 
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Cost Measure 

Frequency 

Percent of 
Sample 

Rank 

$ spent per headcount faculty served per AY 

9 

24 

2 

$ spent per hours of training provided per AY 

10 

27 

1 

$ spent per new online course produced per AY 

10 

27 

1 

$ spent per courses redesigned using technologies per AY 

9 

24 

2 

$ spent per changes made to F2F classes per AY 

2 

5 

3 

$ spent per measure of student learning (e.g. GPA) per AY 

0 

0 

4 


NOTE: AY denotes academic year 


While perhaps the first response to the results in Table 3 is “congratulations” to the institutions using 
various cost-efficiency or cost-effectiveness measures, the second response is to note that only one- 
quarter (10 of 39 total institutions) of respondents are currently doing so. One wonders why so few 
institutions are conducting this type of analysis, how helpful (or not) these measures are to institutions, 
and if there are better measures that provide institutional leaders with the information they need to assess 
the value of faculty developers. (Why in one part of the survey instrument eight institutions indicated that 
they evaluated the cost of training and in another part ten institutions responded positively to these cost- 
benefit measures is unclear. It may simply mean that these measures apply to the entire training function, 
rather than just faculty development for online teaching. In any case, the source of the difference in 
responses is curious.) 

D. Faculty Served 

The results for research question 4 can be found in Table 4, which presents the percent of a) total tenured 
or tenure-track faculty or b) part-time, adjunct faculty served by the number of institutions falling within 
the percent range. Please note that the sample size for this question was 34% smaller (n=25) than for 
Table 3 (n=38). 


Table 4 

Number of Institutions by Percent of Faculty Served at Institution by Type of Faculty (n=25) 


Percent of Total Faculty Served 

Tenured, Tenure-Track Faculty 
(Number of Institutions) 

Part-time, Adjunct Faculty 
(Number of Institutions) 

No answer 

4 

8 

< 20% 

6 

6 

21% to 40% 

3 

5 

41% to 60% 

7 

2 

61% to 80% 

2 

2 

81% to 100% 

3 

2 


Three things may be concluded from the data in Table 4. First, results indicate that one-third of the faculty 
developers responding to the survey serve half of the faculty at their institutions, and one-eighth serve 
nearly all of the faculty at their institutions. Second, several respondents did not know the number of 
faculty at their institution or the number of faculty served (these are reported in the “no answer” above). 
This is likely not a tenable response should an administration ask for information about the 
accomplishments of the unit; faculty developers need to find ways to prove they are serving the institution 
as well as the numbers/percentage of faculty served. Third, there appears to be better penetration (percent 
of faculty served) among tenured and tenure-track faculty than part-time faculty. There are certainly good 
reasons for this, since part-time faculty often have full-time jobs elsewhere or their obligation to 
institutions may not extend to faculty development. Fourth, if institutional budgets for faculty 
development are lean or are cut in the days ahead, achieving even some modest percentage of faculty 
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served is perhaps an achievement, and one that ought to be calculated and shared with administrators. 
This is especially true for the faculty developers who serve 50% or more of the faculty at their 
institutions. While these data do provide insight into the faculty developers’ service to the faculty, without 
cost data, a “per-faculty cost” for training is not possible to calculate. Faculty developers ought to develop 
ways to estimate such costs. 

IV. RECOMMENDATIONS 

Let us first summarize some of the findings above. First, faculty developers appear to be aware of cost 
issues as they contemplate future budget cuts. Given such scenarios, respondents chose to increase low 
cost or low marginal cost training activities and eliminate or scale back more costly activities. Flowever, 
respondents are also aware of the value of training activities, opting more often for activities with high 
value while scaling back those with less value. This makes our faculty developers seem very rational as 
they assess what to do to create the most value with the resources available. It would be fair to conclude, 
based on these modest data, that faculty developers are attempting to respond to the calls for greater 
attention to cost and quality that are currently being issued by state legislators and national leaders. In 
other words, it’s a good start. 

Flowever, stakeholders also want higher education to focus more attention to improving the outcomes of 
teaching and learning; to put this idea in different words, institutions need to produce evidence of student 
learning and timely graduation. To do so, faculty developers need to go another step (or two) to connect 
their current activities to these important outcomes. As attention increases on higher education, all 
portions of institutions will need to explain and justify their worth as they contribute to student learning 
outcomes. 

But there is another concern. It is not clear what data support the decisions made in Tables 1 and 2. Are 
institutions collecting data that will help them make decisions like these or are decisions based on 
informed opinion? Since only one-quarter of respondents collect and report cost measures, how do they 
know—or upon what data—are such decisions based? This is an important set of issues that cannot be 
answered by the current study, but are well worth asking developers in the future. A future line of 
research might not only investigate what data are collected and how they are used, but how decisions 
based on such data have turned out. Did expanding a particular training activity or scaling back specific 
training content affect student learning? These would be essential questions to answer as a way to deal 
with future budget cuts since the second (third or fourth) round of cuts may irreparably harm what 
developers can help make happen. But without the data, how can we argue that cuts will have this (or 
any) impact? As the field gains experience and produces data-based results on how budget matters have 
affected its ability to impact desired outcomes, faculty developers need to share what they are learning at 
conferences and in publications so that others can learn from their experiences. 

Another recommendation to come from this analysis is to urge institutions that are using or exploring the 
use of cost measures (either cost-efficiency or cost-effectiveness measures) to share their experiences. 
This would include not only sharing what they are learning about the usefulness or accuracy of such 
measures, but also evaluate what the measures capture and what they hide. And finally, are there better 
measures to be used? 

The attempt to capture some data on the number or percent of faculty served was not particularly 
successful. Flowever, are there more credible ways to express the concept of how far faculty development 
has reached across the institution? The issue is not simply to report the number of faculty served 
(although several respondents could not provide these data and should be able to do so), but to capture the 
relationship of faculty served to the costs required to do so. In other words, faculty developers need to 
know how to justify what they contribute to an institution. This is not simply an issue of presenting cost 
information but also using such information to make serious decisions about what to do in the future. 
What may be lacking, in the offices of some of these faculty developers, are the data—carefully designed, 
consistently gathered, appropriately analyzed—that helps them improve what they do at a cost that is 
reasonable to the institution. It is not just a simple case of justifying the existence of faculty development 
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at an institution, but also making decisions that are data-based and help guide these units in an uncertain 
future. 


V. APPENDIX 

A. Items in Survey Instrument Used in This Study 


What outcome measure(s) do you use to evaluate training? (Original item contained 18 measures) 

Cost of training Yes No Don’t Know Will Use in Future 


Suppose that you were forced to cut your unit’s 
Enewsletter 
One-on-one 
Hands-on 

Short session (<2 hours) 

Workshop (2-5 hours) 

One-time session 

Multiple sessions (2-5 sessions) 

Many sessions (>5 sessions) 

Summer semester training 
Year-long training 

Consortial (multi-institutional) training 

Webinars 

Online modules 

Peer training 

Train-the-trainer 

Create online course 

Peer review of course 

Use of instructional design students 


budget, how would you adjust your training activities? 


Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 
Expand Keep 


Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 
Scale Back 


Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 

Eliminate 


Do Not 
Do Not 
Do Not 
Do Not 
Do Not 
Do Not 
Do Not 
Do Not 
Do Not 
Do Not 
Do Not 
Do Not 
Do Not 
Do Not 
Do Not 
Do Not 
Do Not 
Do Not 


Offer 

Offer 

Offer 

Offer 

Offer 

Offer 

Offer 

Offer 

Offer 

Offer 

Offer 

Offer 

Offer 

Offer 

Offer 

Offer 

Offer 

Offer 


budget, how would you adjust your training content? 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 


Suppose that you were forced to cut your unit’s 
CMS (course management system) 
Blogs 
Wikis 
Podcasts 

Facebook/Twitter 
Mobile pedagogies 
Advanced topics for experienced 
faculty 

Problem-based learning 
Experiential learning 
Critical thinking 
Use of case studies 
Creating community 
Student learning styles 
Community of Inquiry 
Blended instruction 
Discipline-specific training 
Assessment of student learning 


Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
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Instructional design models 
Research base(s) of online learning 
Externally-provided training. 


Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 
Expand Keep Scale Back Eliminate Do Not Offer 


How do you assess the cost-efficiency or cost-effectiveness of your faculty development program? Do 
you use a measure that is like (or captures a similar concept but may not be exactly the same as) the 
concepts below? ($ are faculty development dollars.) 

$ spent per headcount faculty served per academic year 
$ spent per hours of training provided per academic year 
$ spent per new online courses produced per academic year 
$ spent per changes made to F2F classes per academic year 
$ spent per measure of student learning (e.g., GPA) per academic year 


Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 


Total number (unduplicated headcount of tenured, tenure-track faculty 
At my institution_ 

Those provided faculty development for online teaching in AY 2011-12_ 
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